Automating reflectance confocal microscopy image analysis for dermatological research: a review

Abstract. Significance Reflectance confocal microscopy (RCM) is a noninvasive, in vivo technology that offers near histopathological resolution at the cellular level. It is useful in the study of phenomena for which obtaining a biopsy is impractical or would cause unnecessary tissue damage and trauma to the patient. Aim This review covers the use of RCM in the study of skin and the use of machine learning to automate information extraction. It has two goals: (1) an overview of information provided by RCM on skin structure and how it changes over time in response to stimuli and in disease and (2) an overview of machine learning approaches developed to automate the extraction of key morphological features from RCM images. Approach A PubMed search was conducted with additional literature obtained from references lists. Results The application of RCM as an in vivo tool in dermatological research and the biologically relevant information derived from it are presented. Algorithms for image classification to epidermal layers, delineation of the dermal–epidermal junction, classification of skin lesions, and demarcation of individual cells within an image, all important factors in the makeup of the skin barrier, were reviewed. Application of image analysis methods in RCM is hindered by low image quality due to noise and/or poor contrast. Use of supervised machine learning is limited by time-consuming manual labeling of RCM images. Conclusions RCM has great potential in the study of skin structures. The use of artificial intelligence could enable an easier, more reproducible, precise, and rigorous study of RCM images for the understanding of skin structures, skin barrier, and skin inflammation and lesions. Although several attempts have been made, further work is still needed to provide a definite gold standard and overcome issues related to image quality, limited labeled datasets, and lack of phenotype variability in available databases.


Introduction to Reflectance Confocal Microscopy
For a long time, biopsies followed by histological and microscopic analysis were the gold standard in studying skin structure. Unfortunately, biopsies are invasive and lead to local inflammation due to the damaged cells, which may alter the studied sample and hinder the study of healthy skin. They can also be traumatizing for the patient, particularly when done repeatedly, e.g., for skin cancer monitoring. In certain cases, biopsies can raise ethical questions, such as for cosmetic testing 1 or for the study of healthy infant skin. Therefore, alternatives to biopsies have been developed; these include magnetic resonance, optical coherence tomography (OCT), and reflectance confocal microscopy (RCM). 2 RCM allows for real-time in vivo visualization of the epidermis and the upper parts of the dermis at the cellular level. 3,4 It provides information on the geometrical and topological properties of the observed tissue. 5 It is noninvasive, thus enabling repetitive sampling of the same area without damage. This makes it a technique of choice when studying dynamic changes of the upper parts of skin over time. 6,7 In addition, RCM allows for the quantitative study of skin cellular structures involved in the makeup of the skin barrier. 8 Unfortunately, it is limited by the maximum depth before the signal-to-noise ratio of the image becomes too low to extract any information, but it provides information faster than microscopic analysis of a biopsy sample.
The confocal microscope was first invented by Minsky in 1957. 9,10 Two versions are currently in use for clinical studies: a handheld in vivo skin imaging microscope 11 and a wide-probe RCM. 12 They offer a horizontal resolution of 0.5 to 1 μm and a vertical resolution (optical section thickness) of 3 to 5 μm, to a depth of 150 to 200 μm depending on the observed site. 12 RCM is based on the collection of signals arising from light reflections at the interface of microstructures with different indices of refraction. Such microstructures are cell membranes, melanosomes, collagen fibers, lipidic layers, keratin fibers, and intracellular organelles. 13 The closer the size of the organelle is to the wavelength of the light source and/or the higher its refractive index is compared with its surroundings, the brighter it appears. 2 RCM is an appropriate technology to study structures in vivo, as the energy of the incident light, although sufficient to produce a signal, is too weak to initiate a photobiological process, hence allowing for a visualization of living cells without disturbing or changing their structure or function. By contrast, the collection of a biopsy is accompanied by local inflammatory reaction of the tissue.
RCM stacks are gray-scale images acquired at sequential depths from the skin surface. Their orientation is orthogonal to the vertical sections typical in histopathology [ Fig. 1(a)].
An in vivo reflectance confocal microscope is composed of a light source (commonly a nearinfrared laser), pinhole apertures, lenses, a detector, and a mechanism for beam-scanning 14 (Fig. 1). In the reflectance configuration, the objective lens is used to focus the illumination to a spot on the sample and to collect the reflectance signal. The detection pinhole only passes light reflected from the illuminated point in the sample to the detector and significantly rejects scattered light. By scanning the illumination spot on the sample, an image is reconstructed.

RCM Imaging of the Epidermis
The epidermis is an avascular keratinized stratified squamous epithelium generally comprising four distinct layers. From superficial to deepest, these layers are called stratum corneum (SC), stratum granulosum, stratum spinosum, and stratum basale. In the soles and palms, a thicker epidermis is observed, with an additional fifth layer between the cornified and granular layer called the stratum lucidum.
The majority of cells in all layers below the SC are referred to as keratinocytes, named due to their involvement in manufacturing and storing of keratin intermediate filaments. In contrast to the viable keratinocytes, SC is made of dead but enzymatically active keratinocytes called corneocytes. 15 Throughout the lifetime of a person, these cells are shed and replaced by others from the lower layers. The process starts in the basal layer, where cells are continuously produced (by stem cells and transient amplifying cells), lose their attachment to the basal membrane, and migrate toward the upper layers, while undergoing differentiation toward final cell death in a process called cornification.
RCM can be used to observe the epidermal layers, the dermal-epidermal junction (DEJ), and the upper layers of the dermis, 7 thus allowing for the computation of several quantitative descriptors of skin structure, such as keratinocyte density, number of basal keratinocytes around each dermal papilla, length of DEJ, and circumference of dermal papillae. Measuring such parameters on RCM images enables the quantitative study of skin structures and their evolution over time, for example, as a response to different stimuli. In addition to the geometrical parameters, we can also extract information about the topological organization of the epithelium, for example, the distribution of the number of nearest neighbors to each cell, an important factor in determining molecular exchange rates between neighboring cells. 5 The top slices of RCM stacks represent the SC, which appears as large bright areas forming islands surrounded by dark empty areas [ Fig. 2(a)]. These dark areas are due to grooves called skin microrelief lines, 3 whereas the bright signal in the island structure is due to the high reflectance of keratin. The cells are anucleated dead corneocytes, made primarily of aggregated keratin filaments embedded in a lipid matrix, 15 polygonal in shape, and 10 to 30 μm in size. 2 SC thickness is an important factor involved in skin barrier function. 8,15 The thicker the SC is, the more difficult it is for a noxious substance to penetrate into the viable parts of the epidermis (or equivalently for water to transvers the epidermis and evaporate, potentially leading to tissue desiccation). It is 12 to 208 μm thick depending on the body site. 16 Moreover, corneocytes provide a mechanical strength to the skin surface and are involved in protecting the lower layers against UV radiation, and the lipid matrix is important in maintaining skin permeability. [17][18][19] SC thickness can be calculated by the depth difference of the uppermost and lowest optical sections that contain SC structures.
The stratum granulosum [ Fig. 2(b)] and stratum spinosum [ Fig. 2(c)] are the second and third layers in the epidermis from the skin surface, respectively. They are composed of keratinocytes arranged in a honeycomb pattern in minimally pigmented skin and a cobblestone pattern in heavily pigmented skin 7 [Figs. 3(b) and 3(c)]. In minimally pigmented skin, the cells are characterized by a dark center and grainy cytoplasm due to organelles and microstructures 6 and are surrounded by bright membranes. 2,7 In heavily pigmented skin types, due to the high melanin-content in melanosomes, which gives a strong reflectance signal, we observe bright keratinocytes separated by a dark contour. 20 Viable keratinocytes are found at depths of 20 to 100 μm 21 and are about 10 to 15 μm in size. 2 Cells are typically larger in the granular layer than in the spinous layer, 5 where they have a higher density. Indeed, as the keratinocytes further differentiate while climbing toward the surface, they get wider and flatter.
Toward the basal layer of the epidermis, the cells appear similar in shape but smaller in size compared with the two previous layers [ Fig. 2(d)]. In contrast to the other layers, the basal keratinocytes make a monolayer. These cells are precursors of the keratinocytes in the upper layers and appear brighter than them due to the presence of melanin, which has a high reflectance. 22,23 Melanin is made by melanocytes scattered through the basal layer and then transferred to the keratinocytes. 24 The cells of this layer are adherent to a collagenous membrane that separates the epidermis from the dermis called the basement membrane.
The thickness of the viable epidermis is calculated as the depth difference between the optical sections at which we observe discernable viable keratinocytes in the stratum granulosum and that at which the top of the DEJ appears in the stratum basale.
The undulating DEJ [ Fig. 1(a)] separates the epidermis and dermis and is located at 50 to 100 μm below the skin surface.
Sometimes, bright areas can be observed on RCM images at various layers. They may arise from the keratin in hair shafts [ Fig. 4(a)] or from clustered keratinocytes, also called mottled pigmentation [ Fig. 4(b)]. 1

Skin maturation and aging
Skin function and structure change throughout our lifetime, 25,26 whether this refers to skin maturation during childhood or to skin aging during adulthood. RCM allows us to visualize such changes by imaging the skin of different age groups.
Infant skin is structurally different from that of an adult starting from the appearance of the skin surface to the thickness of the epidermal layers and the extracellular matrix structures in the dermis. On the surface, infant skin has thinner more abundant microrelief lines compared with adult skin. 5,8 This is an important point because the spaces in the microrelief lines may act as reservoirs of topically applied substances, affecting their permeability kinetics. In addition, when comparing RCM images of adults and children, we observe that the SC is 30% thinner in children and the suprapapillary epidermis is 20% thinner. 8,27 The structural differences between infant and adult epidermis translate into functional differences, e.g., when measuring trans-epidermal water loss rates, we notice that it is significantly higher in infants and decreases throughout childhood toward the values recorded in adults. 8 We can also observe on RCM images that, due to higher cell turnover 27 in infant epidermis, keratinocytes and corneocytes are smaller; therefore the cell density decreases with age in both stratum granulosum and stratum spinosum. 5 Both cell area and cell perimeter, which can be measured on RCM images, increase with age, 5 as well as overall epidermal thickness and individual layer thickness. 27 Changes in skin structure do not stop in adulthood as has been documented and quantified in studies using RCM. 28 Early signs of skin aging can be observed, 1 and the effects of cosmetic products can be evaluated. Indeed, in subjects over 70 years of age, Longo et al. observed more irregularly shaped keratinocytes, an increased compactness of collagen fibers under the DEJ, a thinning of the epidermis, and an increase in the presence of clustered keratinocytes appearing as bright spots in RCM images. 1 In addition, although the overall epidermal thickness decreases after 50 years of age, the SC thickness increases. 29 In addition, the number of dermal papillae decreases in aged skin, 30 as does their size, compared with young skin. 29 These changes in skin epidermal structure related to age become more obvious in sunexposed areas where physiological aging related to biochemical processes is accelerated by photo-aging caused by continuous exposure to UV radiation. We can therefore connect structural changes of the epidermis observed with RCM to clinical manifestations of aging, e.g., wrinkles, thinning of the skin, hyperpigmentation spots, and loss of elasticity.
As previously mentioned, RCM can be used in studies requiring repeated assessment of skin structures in healthy as well as diseased skin.

Photoaging
A known risk factor of skin damage, premature skin aging, and even skin cancer is exposure to UV radiations. 31 Because RCM allows us to visualize noninvasively different elements of skin structure at different epidermal layers, it makes the longitudinal studies of skin responses to stimuli such as UV irradiation possible. Therefore, RCM is a reliable technology to assess solar damage. 32 Using RCM images, it has been observed that the SC may appear brighter in sun-exposed areas than in sun-protected areas.
Overall epidermal thickness and keratinocyte density are greater in sun exposed areas and on the face, 6 and their honeycomb pattern organization is often disturbed in sun-damaged regions. 32

Skin inflammatory diseases
RCM has been used to study inflammatory skin conditions, such as psoriasis 33 and allergic contact dermatitis, 17 to evaluate descriptive features of skin inflammation in vivo noninvasively, and to support diagnosis.
Psoriasis is characterized by a thickening of the SC and viable epidermis; both features are quantifiable by RCM. RCM has been used in patients with psoriasis to document thinning of the granular layer, 2 increase in the number and size of dermal papillae, and increase in keratinocytes size and brightness.
Diagnosis of allergic contact dermatitis can be guided by RCM. Some of its characteristics observable in RCM are disrupted SC, vasodilatation, increased epidermal thickness, and detached corneocytes. 17,34 RCM is limited by the maximum depth before the signal-to-noise ratio of the image becomes too low to extract any information. Nevertheless, it provides information faster than microscopic analysis of a biopsy sample and therefore can be integrated as an initial step in clinical diagnosis, 35 e.g., guiding biopsies and determining lesion borders. 36

Skin cancer
Finally, 18 RCM features have been identified as useful in skin cancer diagnosis; two of them, atypical cells, and DEJ disarray, are specific for malignant melanoma (MM) diagnosis. 37 With increasing MM incidence in Europe, RCM-aided MM diagnosis can help with early detection and thus increase survival rates. 38 Using RCM in MM diagnosis may also reduce the number of unnecessary biopsies and benign skin lesion biopsies. 12 In addition to diagnosis, RCM can also be used in the examination of MM and nonmelanocytic skin tumors. Indeed, the large field of view provided by RCM can be used to determine the lesion margins as it can cover a much larger area than classical histopathological approaches, such as biopsies. 2,12 Therefore, it enables the monitoring of tumor growth and response to nonsurgical treatment and potentially guides its surgical excision.

Cosmetology
Biopsies and other invasive methods are rarely used in the study of cosmetic product effect on skin due to ethical considerations. RCM provides a noninvasive alternative that can help to link the changes in structure to changes in appearance, e.g., aging. RCM is useful for assessing the impact of topical formulations on the cellular structure of the skin, e.g., retinoic acid and retinol, 39 and to measure the in vivo kinetics of the skin after application of moisturizers, 40-43 e.g., by measuring epidermis thickness and width of skin folds. Assessment of moisturizer photoprotection efficacy 44 and cleanser efficacy 45,46 is also possible with RCM.

RCM Limitations
All of the previously discussed studies are limited by the maximum depth for RCM imaging, which varies depending on the observed tissue type. RCM use is limited to the epidermis and upper layer of the dermis. Although refractive microstructures are abundant in deeper dermis (e.g., collagen fibers, inflammatory cells), light intensity and coherence drop exponentially with depth. In addition, most studies to date require manual analysis of RCM images, which calls for training that takes 4 to 6 months. 12 The process is tedious and time-consuming and is subject to human error and interexpert variability. Feature extraction could be facilitated by automated image analysis methods. RCM use could then become more widespread 47 in skin research, training time could be reduced, and image analysis could be standardized.

Automatic Identification of Epidermal Layers
Attempts at automatically labeling the four epidermal layers have been made using machine learning approaches. [48][49][50][51][52] The maximum accuracy obtained by these algorithms, i.e., percentage of correctly identified layers against ground truth, was reported to be 88%. 48 Somoza et al. 51 used an unsupervised texton-based approach to achieve 54% classification accuracy. To do so, they generated a library of microstructures called textons by convolving the RCM images by 10 of the Leung-Malik filters, which matched the size of a keratinocyte. They then applied principal components analysis (PCA) to the generated filter-space to reduce the texton-space dimensionality to three, followed by a K-means clustering with 15 clusters. This texton library was applied to test RCM images, and the results were projected on the three PCA axes. Each pixel on the RCM images was classified as one the 15 textons, based on Euclidian distance, and represented as a 15-dimentional texton histogram. This histogram dimension was reduced to three by applying a second PCA. Finally, by applying K-means clustering with five clusters, for the four layers of the epidermis and the dermis, they obtained the classification of each pixel (Fig. 5).
This texton-based approach could be adapted for other classification problems, such as assessing the impact of a treatment on different skin diseases or studying skin maturation and aging. This could be achieved by extending the texton library to include more features. This method, however, could also be improved by including higher level information and features. Indeed, it does not include cellular characteristics or the presence of reflective or darker surfaces that are taken into account by experts manually identifying epidermal layers on RCM images. In addition, this was only a pilot study conducted on three stacks, and it considered that each image contains only a single epidermal layer, which is often not true.
Hames et al. 50 used a bag of features approach to classify RCM images into four categories: SC, viable epidermis, DEJ, and papillary dermis. They established four features, inspired by prior knowledge of RCM images: (1) the presence of a visible honeycomb pattern of viable keratinocytes indicative of the viable epidermis, (2) the presence of bands indicative of basal cell/ dermal papillae indicative of the DEJ, (3) the absence of stratum basale features, and (4) visible papillae indicative of the papillary dermis. Any image before the first visible viable keratinocyte is considered to belong to the SC. Using these features, they built a feature dictionary from small image patches, which they used to represent each test image as the histogram of counts of visible features, and then they classified this histogram with an L1 regularized logistic regression as one of the four categories. They obtained a classification accuracy between 62.9% and 95.6% depending on epidermal layer, body site, and phenotype. Not all phenotypes were included, nor were all of the body sites, and the study did not include diseased skin. The method assumed the presence of a single epidermal layer per image, which is rarely true. In addition, there is no clear pattern explaining the differences in accuracies. Finally, this method does not take advantage of the three-dimensional (3-D) organization of skin to improve the results.
Kaur et al. 49 developed a hybrid deep learning approach for classification of RCM imagers in five categories: the four epidermal layers and the dermis. First, each RCM image was convoluted by a 48-filter bank. Then using a prebuilt texton library, each pixel was represented with a patch centered around it and labeled. Each pixel was then associated to its eight nearest neighbors; thus, each image was represented by eight texton maps. These maps were pooled together by weight while the dark pixels were ignored. Finally, the obtained histograms for each image in the training set were used to train a convolutional neural network (CNN), the parameters of which were determined empirically. They achieved 82% accuracy but only tested the algorithm on three RCM image stacks. In addition, the results are limited by the features in the texton library. Using a multiresolution, multiorientation filter bank to build the library does give more features than if they were determined manually, but it complicates the interpretation of each feature.
Finally, Bozkurt et al. proposed a method to automatically classify RCM images into epidermal layers based on recurrent CNN. 48 They introduced a mechanism called a Toeplitz structure that helps in the interpretation of the model by informing on which image the model's decision was based. This model is an encoder-decoder, with bidirectional recurrent units and Inception V3 53 networks. This approach achieved 88% accuracy in classifying RCM images into epidermis, DEJ, and dermis. This approach was tested on a much bigger dataset than the methods described above. Moreover, it includes higher level information by considering three surrounding images to make a prediction, which is frequently done when manually identifying epidermal layers, but using a CNN leads to a loss of feature interpretability.
Overall, neural network-based approaches 48,49 were more successful in correctly classifying images to epidermal layers than algorithms based on texture analysis, 50,51 but comparison between methods is not straightforward as the datasets used for each method vary in size, from 15 to 504 stacks, i.e., from 1500 to 21,412 images. The methods also vary in types of included samples, with some containing only normal healthy skin and others also including lesional skin. Finally, not all RCM stacks were taken in the same sites, but all were captured using a VivaScope 1500. Table 1 summarizes the methods described for automated epidermal layer classification.

Automatic Identification of the DEJ
The DEJ plays a fundamental role in wound healing and molecular exchanges in the skin. Skin cancer lesions are often characterized by DEJ structural changes. In addition to applications related to skin cancer, rapid localization of DEJ would be helpful in the study of healthy skin structure, for example, related to aging. Such approaches have been tested to delimitate the DEJ in RCM images, with varying levels of accuracy. Indeed, the DEJ is harder to delineate in lighter pigmented skin due to the lack of strong melanin signal. Some researchers have considered the DEJ identification to be a segmentation problem, whereas others look at it as a classification problem. Kurugol et al. 52 achieved 88% accuracy in more pigmented skins and 60% accuracy in minimally pigmented skins, with an average distance from the expert labels of 7.9 and 8.0 μm, respectively, using a bag of features approach with support vector machine (SVM) algorithms. Indeed, Kurugol et al. 52 started by applying to each image a skin type detection algorithm based on the detection of bright cells in the stratum basale, which is characteristic of heavily pigmented skin. On RCM images of heavily pigmented skin, they clearly detected the DEJ, but for images of minimally pigmented skin, where melanin contrast is lower, a transition zone between dermis and epidermis was detected instead.
For RCM images of heavily pigmented skin, where the image contained bright basal cells and had one strong intensity peak, they relied on intensity information to determine DEJ. If the image had more than one intensity peak, then they used a texture-based detection algorithm. For minimally pigmented skin, Kurugol et al. applied an SVM classifier trained on features from manually labeled images. These features were automatically selected from a list of 170 computed parameters and were the most discriminative and less redundant in the training set (Fig. 6). This approach showed better results on heavily pigmented than on minimally pigmented skin. On minimally pigmented skin, the approach failed to identify the DEJ where wrinkles are present, limiting its use. In addition, Kurugol et al. mentioned a cross-expert correlation of 81% in DEJ delineation on minimally pigmented skin, highlighting the difficulty of this task but also showing that their automated method is not on par with manual identification of the DEJ by experts.
Robic et al. 54,55 used a random forest classification combined with spatial regularization based on a 3-D conditional random field 54,55 (CRF) and achieved 54% accuracy in identifying the DEJ and 90% and 75% accuracy in identifying the epidermis and dermis, respectively. For each pixel in an RCM image of light pigmented skin, they predicted the probability of it belonging to one of three categories, i.e., dermis, epidermis, and uncertain, with a random forest classifier. These results were fed to the CRF to predict the labels of the pixel neighbors by imposing the transition order of the epidermal layers (Fig. 7). This method was not tested on RCM  images of heavily pigmented skin, but it performed on par with state-of-the-art methods [48][49][50]52 on minimally pigmented skin. It took into account the possibility of different layers per image, which other methods do not, as they performed classification at the image level and not at the pixel level.
Bozkurt et al. proposed a method 47 to automatically delineate DEJ based on recurrent CNN with attention to aid cancer diagnosis. This approach achieved 88% accuracy. It is based on the knowledge that skin maintains a strict sequential ordering: epidermis, DEJ, and dermis so recurrent CNN are appropriate to automatically identify epidermal layers on RCM images. Bozkurt et al. first trained a deep CNN and then augmented it with recurrent layers, so the model would take into consideration information from the image and its surrounding slices. The CNN model used was an Inception v3 network. 53 This method was tested on a much bigger dataset than other approaches, which allowed the authors to take into consideration the dependencies between sections, i.e., order of the layers, accounting for the 3-D organization of the skin, which helps circumvent the difficulty of identifying the DEJ by giving information on the DEJ shape across different images. However, the complexity of the model and the size of the training set hindered the training computational performance.
Finally, a Poisson point process 56 approach managed to identify the DEJ with an average error of 5.4 μm in heavily pigmented skin and of 12.1 μm in light pigmented skin. It is an unsupervised generative framework that segments and detects complex objects, based on prior knowledge of the shape. Here, the DEJ was modeled as a succession of an unknown number of hills at random locations determined by a marked Poisson process. This method is based on the assumption that prior shape information improves object detection and thus provides an explicit model of the boundary, whereas other methods rely on extracted features. However, it requires parametrization of the Poisson point process based on prior knowledge, which in turn requires detailed labeling of the images. Therefore, the parameters in this approach can easily be interpreted, which is often not the case with deep learning-based approaches. 47,55 On the other hand, this method relies on a small number of structural and physiological features that are meaningful to the expert, which makes it reliable but could lead to a loss of information related to other characteristics, sometimes even determined by the machine but unknown to the expert.
All of the methods used images acquired with a VivaScope 1500 microscope, but not all images were captured at the same body site. Table 2 summarizes the methods described for DEJ delineation.

Automatic Identification of Pigmented Skin Lesions
Automated identification of lesions from RCM images has also been investigated. We distinguish two types of applications for these algorithms: (1) finding melanoma patterns and (2) distinguishing nonmelanocytic lesions from melanoma.
For the first application, an algorithm to identify melanocytic lesions on RCM images based on wavelet transform obtained moderate success with 55% of the melanomas and 47% of the benign nevi being correctly identified. 57 Another approach aimed to reproduce clinician analysis of an RCM image to determine the presence of melanoma 58 by identifying patterns in the DEJ mosaics and classifying them into melanoma or nonmelanoma with a sensitivity of 55% to 81% and a specificity of 81% to 89%. A more recent approach by Bozkurt et al. 59 used a multiresolution convolution neural network to identify similar patterns as Kose et al. 58 It achieved 95% average specificity and 77% average sensitivity.
For the second application of distinguishing between melanomas and nonmelanocytic lesions, Halimi et al. 60 proposed a Bayesian model to quantify RCM images reflectivity and classify images into two categories, healthy and lentigo patients, based on their reflectivity distribution. They obtain an accuracy of 98%.
Zorgui et al. 61 obtained similar results, with an accuracy of 98% with a CNN. The CNN was trained on normalized resized RCM images with a pretrained Inception V3 model. 53 Transfer learning was then used to apply the model to skin RCM images. 62 Table 3 summarizes the methods described for skin lesions identification. Finally, Bozkurt et al. 59 proposed a CNN inspired from the U-net 64 architecture to identify six classes: nonlesion, artifact, meshwork pattern, ring pattern, nested pattern, and aspecific/patternless. Indeed, this model was built on a dataset containing RCM images of both lesional and nonlesional skin. This model slides through RCM images with a sliding window with 75% overlap and applies three consecutive nested U-nets. This generates segmentations at different resolution levels. Each U-net model generates a probability map. The deepest U-net model only takes a sliding window as input, whereas the others use a concatenation of the upsampled probability map at the higher level and sliding window. This model achieved 73% classification accuracy.
The goal here is not to compare the performance of these approaches, as they do not all identify the same lesion, but to give an idea of what can be achievable with RCM images.

Automatic Identification of Cells
RCM individual cells provide important information in the assessment of skin health, but their manual identification is tedious, time-consuming, and subject to expert interpretability. Very few attempts have been made to automatically identify individual cells or nuclei in skin RCM images.
Harris et al. 65 proposed a pulse coupled neural network to automatically segment nuclei in oral mucosa RCM images. Identifying the nuclear-to-cytoplasm area ratio is a useful indicator in the early diagnosis of cancer. Unfortunately, RCM images tend to be noisy and nonuniform, which makes the development of an accurate segmentation algorithm complicated. 66,67 RCM images in which the background was removed were filtered, and a spiking cortical model 68 was applied to them, followed by an artificial neural network classifier, which outputs a segmented image. The approach had 90% accuracy (trained using eight images and tested on 28). The small size of the training set may impact the accuracy of the model. It would be difficult to apply this approach to skin RCM images that are noisier than images of the oral mucosa and riddled with small organelles within the cytoplasm and membranes of similar shape and size than a nucleus.
Gareau 69 attempted an automated identification of keratinocytes. An error function reflectance profile was trained on labeled RCM images and then tested on other images to identify keratinocytes coordinates. All images belong to the same stack. The obtained keratinocyte density matched prior knowledge based on manual counts. The model supposed that the keratinocytes center is darker than the rim. This assumption fails on basal cells due to brightly scattering melanosome caps over the nuclei. The method may be improved by training two separate models for the granular and spinous layers as their cells differ in size. It is also unclear how it behaves when tested on RCM images of other people of different ages, as keratinocytes size changes with age. In all models described above, results differed between minimally and heavily pigmented skin or were not tested in both cases. Any algorithm developed for the analysis of RCM images should be tested on skin types with various degrees of pigmentation.

Discussion
Various attempts have been made at automating the analysis of RCM images in dermatology, from automatic epidermal layer classification and DEJ identification to lesion detection and cell identification. The published work to date shows that there is high potential in the application of machine learning and/or image analysis algorithms to RCM images.
The presented algorithms have various levels of success and accuracy, but a direct comparison is difficult as used datasets varied in size and skin samples varied in sampled body site, volunteer age, and phototype. Overall, the application of computational methods in RCM is made difficult by poor image quality, high noise, and low contrast. In addition, training any supervised model requires the manual labeling of the images to obtain a ground truth, which is time-consuming and tedious and, with relative variability between experts, highlights even more the need for automation in RCM image analysis. Furthermore, the used datasets often lack in variety in terms of subject ages and skin phototypes, which introduces bias to the models and reduces their general application to all populations.
Another noninvasive in vivo technology used in the study of skin is OCT. OCT has been used in the study of different skin lesions, especially carcinomas and inflammatory skin diseases. OCT has a greater imaging depth than RCM of up to 2 mm, but its resolution is limited and does not allow for identification of individual cells. Attempts at automating epidermal layer classification, hair follicle identification, lesion detection, and skin inflammation have been made for OCT images, but similar to RCM, a generally accepted gold standard does not exist.

Conclusion
RCM can provide a quantitative evaluation of skin barrier physiology and how it changes due to age or responding certain stimuli, with near histological resolution. However, the study of RCM images is currently mainly done manually and therefore is tedious, time consuming, and subject to human interpretation. An automated approach to extract quantitative descriptors from confocal images would enable an easier, more reproducible, precise, and rigorous study of these images. Although attempts at the automation of descriptor extraction have been made, a globally accepted gold standard that combines all approaches and can be used by biologists and clinicians is still an open problem. Therefore, future research should focus on methods allowing for an easier translation of images into relevant quantifiable parameters and on making RCM easier, faster, and more accessible to use.

Disclosures
The authors have no relevant financial interests in this article and no potential conflicts of interest to disclose.
France. She received her MEng degree in bioinformatics and modeling from INSA Lyon, France, in 2019. Her research focuses on applying image analysis, machine learning, biostatistics, and omics analysis to study skin.

Georgios Stamatas is a research associate director, Translational Science at Johnson & Johnson.
His research focuses on method development and applications on understanding skin physiology and topical product effects. He works on the differences between pediatric and adult skin and has transformed our understanding of newborn and baby skin maturation. He received his PhD in chemical/biomedical engineering from Rice University, has coauthored close to 100 scientific publications, and holds several patents.
Xavier Descombes received his PhD in computer science in 1993, his master's degree in mathematics in 1990, and his engineering diploma in 1989. He is currently heading the Morpheme team. He has pioneered the marked point process approach in image processing and currently focuses on image processing for biological applications. He received the "Prix de la Recherche, Catégorie Santé" in 2008. He has published more than 50 publications in international journals (H-index 39).